Method and apparatus for transferring data within viewable portion of video signal

ABSTRACT

A video signal has a visual image ( 12 ) and carries data for optical detection ( 54 ) via the image. The data for optical detection is encoded ( 52 ) as data bits in a plurality of lines within a defined region ( 14 ) within the image. A corresponding decoder ( 16, 58 ) uses maximal energy methods to detect the defined region prior to decoding the data.

RELATED PATENT APPLICATION

This application is a National Phase Application of PCT/IL03/00689having International Filing Date of 19 Aug. 2003, which claims priorityfrom U.S. Provisional Patent Application No. 60/404,525 filed 20 Aug.2002.

FIELD AND BACKGROUND OF THE INVENTION

The present invention relates to a method and apparatus for transferringdata over the viewable portion of a video signal, and more particularlybut not exclusively to a method and apparatus that provideselectronically downloadable data obtainable by optical means from avideo screen.

The most common approach to applying data to a video signal currently inuse is to insert data during the vertical blanking interval, such as inthe generation of closed captioning signals. A major disadvantage ofthis method is that data in the vertical blanking interval can beremoved or changed by the TV broadcaster, usually quite unintentionally,as part of the incidental operation of the broadcasting infrastructure.In certain cases the same program may be broadcast simultaneously inanalog and digital formats, the data being preserved in the analogformat but lost say in the digital format. Placing the data within thevisual signal overcomes the problems introduced by incidental operationof the broadcasting infrastructure and thus allows a program producer tointroduce data into his signal with confidence that it will be availableto viewers. The program producer does not need to require thebroadcaster to make changes to his broadcasting infrastructure.

An additional disadvantage of embedding data in the vertical interval isthat decoding requires access to the video signal and is not possibleoptically using the light transmitted by the TV set. Data decodingtherefore requires changes to the TV set or to the Set Top box.Embedding of the data into the screen, image ensures that data transferdoes not require any modifications to the end user equipment.

In order to permit optical detection, a number of patent applicationstherefore teach methods of placing data on the clearly visible portionof the video signal. One advantage of the latter approach is it does notrequire any changes to the broadcasting infrastructure or end userequipment. Patent number WO 01/03434 A1 to Gamut Interactive describes amethod for embedding data on the visible portion of a video signal. U.S.Pat. No. 6,094,228 to Koplar Interactive describes an additional methodfor invisibly embedding data on the visible portion of a video signal.Both approaches have been tried in the past but the methods as disclosedare limited by a narrow bandwidth, and are thus unable to supporttransmission of large quantities of data from the TV screen to theuser's device. In practice these approaches limit possible applicationsof the technology to the transmission of a single code or a singleidentification number. The commercial coupon data itself is pre-storedin the memory of a handheld device provided to the user, and the signalembedded in the video simply triggers the appearance thereof.

International Patent Application no. WO 01/93473 A2 “Systems and Methodsfor Distributing Information Through Broadcast Media” to the presentapplicant, the contents of which are hereby incorporated by reference,discloses a method and corresponding apparatus for distribution ofelectronic messages that are embedded in a TV video signal. The messagesmay for example be data that allows printing out of coupons related tocommercial material currently being broadcast. The disclosure teachesoptical signals that can be embedded within the screen view and furtherdiscloses a point and click device that can be pointed at the screen todetect the signals optically. In one embodiment the point and clickdevice includes a built-in printer. Whilst the disclosure provides amajor improvement in terms of bandwidth over the prior art, it leavesopen questions of the best technical method for embedding data withinthe video screen image in order to minimize noise, maximize bandwidthand provide the data in the most convenient and reliable manner possibleto the user.

SUMMARY OF THE INVENTION

It is a general object of the present invention to overcome the abovedisadvantages and to enable high enough bandwidth, and minimal noise fortransmitting of commercial coupons and the like to the handheld devices.It is further an object to provide the data in a manner most convenientto the user. Thus, given that a user is not going to point his device atthe screen at precisely the moment the program producer wants him to, itis necessary to provide the signal with a certain robustness to the timeat which the user points the device.

Furthermore the user may not necessarily keep the device pointed in theright direction for the entirety of the duration needed to download thecomplete signal but may point, waver and then point again.

According to a first aspect of the present invention there is provided avideo signal having data encoding a visual image over successive frameseach frame comprising scan lines of pixels, said video signal carryingdata for optical detection via said image, wherein said visual imagecomprises at least one defined region and said optical data is encodedwithin scan lines within said defined region.

Preferably, the defined region is substantially rectangular.

Preferably, said defined region comprises a plurality of substantiallyrectangular regions.

In one embodiment, said data for optical detection comprises a pluralityof messages, each message having its own predefined rectangle whichpersists over successive frames for the duration of the respectivemessage.

Preferably, said data for optical detection comprises a message, andsaid message is repeated cyclically over successive groups of frames.

In one embodiment, said data for optical detection comprises at leastone position flag to indicate places in said cyclical repetition.

In one embodiment, said data for optical detection comprises asynchronization field to provide orientation within said cyclicalrepetition.

In one embodiment, said data for optical detection is arranged into aplurality of separate regions, thereby to increase message capacity.

The signal may further comprise an audible signal to indicate thebeginning of said data for optical detection.

The signal may further comprise an audible signal to indicate the end ofsaid data for optical detection.

The signal may further comprise a second audible signal to indicate theend of said data for optical detection.

In one embodiment, borders of said region are defined according to arequired capacity of a message being carried by said data for opticaldetection.

Preferably, said data for optical detection comprises a plurality ofdata symbols, and each symbol is encoded in at least one of said linesof said series.

Preferably, said data for optical detection comprises a plurality ofdata symbols, and in which each data symbol is encoded in two of saidlines of said series.

The signal may further comprise error correction encoding.

Preferably, said data for optical detection is superimposed, within saidregion, over an underlying image.

In one embodiment, said superimposing comprises blending with saidunderlying image.

In one embodiment, said rectangle is defined by a surrounding frame.

In one embodiment, said data for optical detection is modulated intosaid image by at least one of a group comprising black and whitemodulation, and color modulation,

In one embodiment, said data for optical detection is modulated intosaid image by at least one of a group comprising frequency shift keying,and quaternary frequency shift keying.

Preferably, said data for optical detection comprises at least one of agroup comprising time information and program information.

Preferably, said data for optical detection comprises a software updatefor a decoding device.

According to a second aspect of the present invention there is providedapparatus for encoding data for optical detection within the visualimage of a video signal, the apparatus comprising:

-   -   a defining unit for defining a region within said image to carry        said data for optical detection, and    -   an encoder, associated with said defining unit, for encoding        said data for optical detection into video scan lines within        said region.

Preferably, said region is retained over a succession of video images.

Preferably, said encoder is configured to superimpose said data foroptical detection over an image, and wherein said superimposing isconfined to said region.

Preferably, said superimposing comprises blending into said image.

In one embodiment, said blending comprises invisibly blending, such thatsaid data for optical detection is substantially invisible to a user.

Preferably, said region is substantially rectangular.

Preferably, said defined region comprises a plurality of substantiallyrectangular regions.

Preferably, said defining unit is configured to surround said regionwith a frame.

In one embodiment, said encoder is configured to encode said data foroptical detection by at least one of a group comprising black and whitemodulation, and color modulation,

In one embodiment, said encoder is configured to encode said data foroptical detection by at least one of a group comprising frequency shiftkeying and quaternary frequency shift keying.

In one embodiment, said data for optical detection comprises a printfile configured for printing.

In one embodiment, said data for optical detection comprises barcodedata.

Preferably, said data for optical detection comprises at least one of agroup comprising time information and program information.

In one embodiment, said data for optical detection comprises a softwareupdate for a corresponding decoding device.

Preferably, said encoder is configured to insert said data as symbols,each symbol being encoded within at least one video scan line withinsaid region.

Preferably, said encoder is configured to insert said data as symbols,each symbol being encoded within at least two video scan lines withinsaid region.

Preferably, said encoder is configured to insert said data for opticaldetection in cyclical repetition.

Preferably, said data for optical detection comprises at least oneposition flag to indicate places in said cyclical repetition.

Preferably, said data for optical detection comprises a synchronizationfield to provide orientation within said cyclical repetition.

According to a third aspect of the present invention there is providedapparatus for decoding data encoded optically within a defined region ofa visual image of a video signal, said apparatus comprising:

-   -   an optical detector for optical detection of said image,    -   an encoded region determination unit for determining, from        output of said optical detector, boundaries of said defined        region within said scanned image, and    -   a data decoder associated with said encoded region determination        unit for decoding data received at said optical detector that is        determined to be within said defined region.

In one embodiment, the apparatus for decoding may be incorporated withina mobile device.

In one embodiment, said mobile device is a handheld device.

The apparatus preferably comprises a printer associated with an outputof said data decoder, for printing out decoded data.

In one embodiment, said data for optical detection comprises anidentification flag and said printer is configured to make only apredetermined number of printouts per identification flag.

In one embodiment, said predetermined number is one.

In one embodiment, said mobile device is incorporated within a mobiletelephone.

In one embodiment, the apparatus is incorporated within an accessory fora mobile telephone.

In one embodiment, a software updater is included for using said decodeddata for self-updating.

Preferably, said data for optical detection repeats cyclically and saiddecoder is operable to decode data whenever a substantially fill cyclehas been detected irrespective of where in said cyclical repetition saiddecoding starts at.

In one embodiment, said decoder is operable to use error correction datawithin said data for optical detection to deduce a starting pointthereof.

Preferably, said data for optical detection comprises at least oneposition flag to indicate places in said cyclical repetition.

Additionally or alternatively, said data for optical detection comprisesa synchronization field to provide orientation within said cyclicalrepetition.

In one embodiment, said encoded region determination unit comprises anentropy summation unit for obtaining summations of entropy over a videoimage to identify said region as a region having maximal entropy.

In one embodiment, said entropy summation unit is configured to providea sliding window to move over said image, to calculate an entropysummation for each window position and to identify said region as awindow position having a maximal entropy.

In one embodiment, said encoded region determination unit is configuredto provide relatively large changes in position of said sliding windowbetween each entropy summation to provide coarse determination of saidregion.

In one embodiment, said encoded region determination unit is configuredto provide perturbations to said coarse determination of said region toachieve fine determination of said region.

In one embodiment, said encoded region determination unit is configuredto enable decoding of scan lines within a perturbation range around saidregion, so that data extracted therefrom can be used if said scan linesare subsequently determined to be within said region.

In one embodiment, said encoded region determination unit is configuredto use a mean least squares (MLS) to achieve fine determination of saidregion.

In one embodiment, said entropy summation unit is configured to summateentropy over substantially all scan lines and all frequencies withinsaid image.

Preferably, said encoded region determination unit is operable tocontinue to use a region as detected in previous frames of said image.

Preferably, said data for optical detection comprises symbols encoded asfrequencies within said scan lines, said data decoder being operable todeduce said symbols from said frequencies.

Preferably, said data for optical detection comprises symbols, encodedin a plurality of scan lines within said region.

In one embodiment, said data decoder is operable to sum a respectiveplurality of scan lines, and to decode a corresponding signal from saidsummation.

The decoding apparatus may be configured for scanning image produced bya plurality of video encoding methods, said apparatus being operable toscan for a first method, and if energy detected falls below apredetermined minimal threshold then to restart the scanning process.

A variation of the above is configured for scanning image produced by aplurality of video encoding methods, said apparatus being operable toscan for a first method, and if energy detected falls below apredetermined minimal threshold then to scan for another one of saidplurality of methods. Preferably, the variation is configured to retaina last used video encoding method as a default method for initialscanning.

In one embodiment, said decoder is configured to correct data decodingaccording to subsequently carried out perturbations.

According to a fourth embodiment of the present invention, there isprovided decoding apparatus for decoding cyclically repeating data, saidapparatus comprising:

-   -   a decoder for applying a decoding procedure to said data,    -   data handling logic for controlling said decoder and for        outputting decoded data, and    -   a starting position recognizer associated with said decoder for        using an output of said decoder to determine a start position of        said data, said start position being used as a parameter for        said data handling logic so as to ensure that said cyclically        repeating data is decoded and output in a correct sequence from        said start position.

Preferably, said start position recognizer is configured to recognize asynchronization field within said cyclically repeating data.

In one embodiment, said start position recognizer is an error correctioncircuit for operating with a cyclic redundancy code.

Unless otherwise defined, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. The materials, methods, andexamples provided herein are illustrative only and not intended to belimiting.

Implementation of the method and system of the present inventioninvolves performing or completing selected tasks or steps manually,automatically, or a combination thereof. Moreover, according to actualinstrumentation and equipment of preferred embodiments of the method andsystem of the present invention, several selected steps could beimplemented by hardware or by software on any operating system of anyfirmware or a combination thereof. For example, as hardware, selectedsteps of the invention could be implemented as a chip or a circuit. Assoftware, selected steps of the invention could be implemented as aplurality of software instructions being executed by a computer usingany suitable operating system. In any case, selected steps of the methodand system of the invention could be described as being performed by adata processor, such as a computing platform for executing a pluralityof instructions.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is herein described, by way of example only, withreference to the accompanying drawings. With specific reference now tothe drawings in detail, it is stressed that the particulars shown are byway of example and for purposes of illustrative discussion of thepreferred embodiments of the present invention only, and are presentedin the cause of providing what is believed to be the most useful andreadily understood description of the principles and conceptual aspectsof the invention. In this regard, no attempt is made to show structuraldetails of the invention in more detail than is necessary for afundamental understanding of the invention, the description taken withthe drawings making apparent to those skilled in the art how the severalforms of the invention may be embodied in practice.

In the drawings:

FIG. 1 is a simplified block diagram showing detection of data within avideo display according to a preferred embodiment of the presentinvention;

FIG. 2 is a simplified flow diagram illustrating encoding of data into avideo image according to a preferred embodiment of the presentinvention;

FIG. 3 is a simplified flow diagram showing decoding of data from asequence of video images according to a preferred embodiment of thepresent invention;

FIG. 4 is a simplified block diagram illustrating a decoding deviceaccording to a preferred embodiment of the present invention; and

FIG. 5 is a simplified block diagram illustrating simplified encodingand decoding devices according to an embodiment of the presentinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present embodiments show a method and apparatus for encoding anddecoding data within the visual field of a video signal, in a mannerwhich is more robust to use conditions and provides better signal tonoise ratio than the current art.

Before explaining at least one embodiment of the invention in detail, itis to be understood that the invention is not limited in its applicationto the details of construction and the arrangement of the components setforth in the following description or illustrated in the drawings. Theinvention is capable of other embodiments or of being practiced orcarried out in various ways. Also, it is to be understood that thephraseology and terminology employed herein is for the purpose ofdescription and should not be regarded as limiting.

Referring now to the drawings, FIG. 1 illustrates data encoded in avideo signal and an optical detection device according to a firstpreferred embodiment of the present invention. A television or likevideo screen 10 displays a moving image as a sequence of frames eachdisplaying temporal versions of visual field 12. Each frame comprises aseries of scan lines and each scan line comprises a sequence of pixels.A region 14 is defined within the visual field and the parts of the scanlines that are enclosed within the defined region 14 carry data. Thedata is detected optically by optical decoding device 16 which islocated remotely from screen 10 but pointed at the screen in such a wayas to provide line of sight to the screen.

Decoding device 16 is preferably a mobile device that interprets datavisually embedded in a video signal. In one embodiment the data is aprint file and the data is program related data added to a broadcasttelevision signal. The mobile device preferably contains a small printer18 which can be used to print out ticket sized items. Typicalapplications include coupons related to commercial broadcasts or torecipes for cookery programs or to the lyrics for songs and the like.The print file may follow a user recognizable audible tone, which primesthe user to point the decoding device 16 at the screen and obtain thedata, for example by pressing a download button. The download button maycause the device to detect and decode the coupon data. In oneembodiment, the initial download displays the data on a screen and thena second press of the same or another button causes the file to beprinted.

Printer 18 is preferably a miniature printer, and may be integrated intothe mobile device as shown in the Figure. The printer may for example bea thermal or an impact printer. The printer prints out text, an optionalbarcode, which might be similar to the one used in existing newspapercoupons, and bitmap images such as Logos. The printer may be color orblack and white.

If a printer is used, pressing the download or print button as describedabove results in printing out a paper coupon, which may later on beredeemed at a point of sale in a way similar to existing newspapercoupons.

Alternatively, the mobile device can be connected or incorporated into amobile telephone. The decoded coupon may be inserted into the mobiletelephone memory and may subsequently be used for redemption at pointsof sale or for direct purchasing via the cellular network.

One embodiment enables an advertiser to limit the maximum number ofcoupons that can be printed out for the same promotion. Therefore, aunique identification number may be added to the coupon data to enabletracking of the number of times a certain coupon is printed and toprevent a given device from printing when the maximum is reached, or toprevent redemption at points of sale of more than a predeterminednumber. It is further possible to add a maximum permitted number ofprintouts as an additional field within the coupon data, thus allowingmore flexible control of any limits per specific message.

Use of identification data with the coupon further enables an advertiserto track the effectiveness of specific advertising routes. Thus, time,TV program or channel and like source identification may be embedded inthe coupon data for gathering upon coupon redemption and later on usedfor advertising effectiveness analysis.

It is further possible to use the same data distribution method toprovide software updates of the device without being required to collectthe devices or to send maintenance personnel to consumers' homes.Therefore, in the present invention software updates may be downloadedvia TV video signal, in a way analogous to message reception. Theconsumer may be asked, say via a message he downloads, to set the TVtuner to a specific channel which is specifically used for softwareupdates and to activate the device when pointed to the Television.

In addition, the data download device 16 may be used as an accessory oras an integrated module in a third party consumer device. The consumerdevice is then able to download software updates via TV video signal. Inthis way new games or ringtones can be added to mobile telephones andlike devices, or new speeches can be added to talking dolls or newmovements added to moving robots etc.

In the case of digital broadcast, the image is transmitted as abitstream. In one embodiment, the image that is transmitted includes theembedded data. In an alternative embodiment, the message data istransmitted as a bitstream along with the video data and is thenembedded into the image by the set top box. The embedded video data,produced in this way by the set top box, is then displayed by the TV asbefore and captured (and printed) by the mobile device.

Reference is now made to FIG. 2, which is a simplified diagramillustrating a procedure for incorporating a data file into the visiblepart of a video field and then subsequently incorporating the videofield into an overall video signal according to a preferred embodimentof the present invention. In a stage S2, a message file is obtained. Ina stage S4 the data is converted into a stream of bits.

In a stage S6 error correction encoding is added to the bit stream, andsuitable redundancy check schemes will be discussed in greater detailbelow. The individual bits of the bit stream are then modulated onto thescan lines that make up a typical image representation in a videosignal, in stage S8, and then in stage S10 the scan lines are writteninto defined regions within successive video frames. The defined regionis typically a rectangle, or window, and in one embodiment the modulateddata overwrites successive lines in the rectangle. When the final lineis reached in a rectangle then the data overwriting continues with thefirst line of the same rectangle in the next frame. Decoding accuracycan be improved by allowing the same data to persist for more than oneframe, in which case writing continues with the next group of frames.

Preferably, encoding of data into the video signal should be possibleboth in a real time working environment and in advance off-linepreparation.

A number of modulation schemes are possible. In one possible modulationscheme, each sequence of bits is represented by a specific frequency outof a predefined set of frequencies. In a preferred embodiment, thelength of the sequence is 1 bit and the set of frequencies includes twofrequencies, one represents ‘0’ and the other represents ‘1’ (FSKmodulation). Thus a scan line where the intensity varies between highand low at one rate is read as a one, and a scan line where theintensity varies between high and low at a different rate is read as azero. However, other alternatives can be selected such as a set of fourfrequencies that represents a sequence of two bits (QFSK) and the like.

In a preferred embodiment, a square wave pattern is used to representthe bits. However, it should be noticed that other wave patterns, suchas sine, could be used as well.

In the present embodiments, data is blended into pre-defined windowswithin the video frame. The size and location of the windows remainconstant for a specific message insertion. For example the size of thewindow may be selected in accordance with the quantity of data and thelocation may be selected to minimize adverse effects on the scene beingviewed. Different sizes and locations can thus be used for variousinsertions. In a preferred embodiment, the number of data windows isone. However, in order to enlarge the data bandwidth it is possible touse more than one data window simultaneously.

As explained above, a data window comprises a plurality of video scanlines, each modulated with data in an optically detectable manner. Whenthe region is rectangular, all of the data lines are substantially equalin length. In FSK, a data line is modulated with a specific frequencysignal. In a preferred embodiment two sequential lines represent eachbit. However, in other embodiments a bit could be represented by asingle line, to achieve higher baud rate, or represented by more thantwo lines, to achieve better SNR (Signal to Noise Ratio).

In a preferred embodiment, the modulated bits replace, that isoverwrite, the original video signal. Overwriting achieves maximum dataenergy. However, it is also possible to add the value of the modulatedsignal to the value of the original video signal, in a process known asalpha blending. Doing so, while reducing the intensity of the modulateddata, is useful for reducing the visual interference of the data to theoriginal video. By reducing the intensity of the modulated data to avery low amplitude, the data can actually be invisible to the normalviewer, whilst still being detectable by an optical device.

In a preferred embodiment, the data is modulated equally over thevisible color spectrum without emphasizing any specific color (i.e.black and white modulation). However, in additional embodiment colorscould be integrated in the modulated data. White sections, as anexample, could be entirely or partially replaced by Blue sections. Blacksections could be entirely or partially replace by Red or Greensections. The purpose of the colors could be either to improve thevisual appearance of the data or to improve the detection of the data.Improved detection can be achieved if a sensor that is more sensitive toa certain color is then used to detect the signal. In addition,detection may be improved if the TV display emits more energy inresponse to a predetermined specific color.

In a preferred embodiment, no frame surrounds the data window. However,it is possible to surround the data window with a frame. The frame maybe of a specific color or specific pattern or any combination of theabove. The purpose of having such a frame may be either to improve thevisual appearance of the data or to improve detection of the data.

Stage S6 comprises the insertion of an error correction code into thebitstream. The correction code allows subsequent decoding to overcome acertain level of bit errors and therefore increases the tolerance of thesystem to noise. Different embodiments may improve the baud rate by notusing error correction, but at the cost of decreased tolerance to noise.

In a preferred embodiment a (850,485,79) binary BCH code is used. Themaximum number of data bits in such as code is 485, and 365 additionalredundant bits are added to provide a code in which up to 39 errors canbe corrected. It is possible, however, to use a different errorcorrection code or use the same code with different parameters. Furthermore, it is possible to use the (850,485,79) BCH code for sending lessthan 485 data bits. In this case both the encoder and decoder assumethat the non-used bits have a constant value, such as ‘0’, which isknown in advance. The non-used bits, in such a case, are not actuallyembedded in the video signal.

In the present embodiment, data is sent in packages. Each group of bits,which may include both data and error correction bits, is sent as asingle package. A short package may be inserted into a single field orwindow. Longer packages are usually inserted over more than one field,or more than one video frame. In general, it is possible to encode andinsert into the video stream either a single package or multiplepackages.

In a preferred embodiment, coupon information is encoded in one datapackage and the package is inserted several times into the video streamin a cyclic manner. Thus, the first field of a new package instance isrebroadcast directly after the last field of the previous instance ofthe same data. Thus if the user fails to point his decoder at the streamat the very start of the data, it is still possible to download acomplete cycle of the data. Furthermore, provided that the decoder isable to identify the start of the data correctly and process it in thecorrect order, it does not matter what order the data is scanned in.Thus the decoder can begin to scan data from the end of the cycle andcontinue from the beginning, and nevertheless decode and present thedata in the correct order. Hence, the detection system is able tocommence scanning the cyclic data immediately and does not need to waitfor the beginning of a new package before detection can start. This isuseful since the user may have limited patience to point the device, ormay not hold the device steady for a long period of time.

In a preferred embodiment, the data package, for example an advertisingcoupon, lasts for 26 video fields. Since the data fields are insertedcyclically, the original message can be reconstructed out of anysequential 26 fields, no matter which field was the first to bedetected. However, a decoder is preferably able to determine where themessage starts in order to decode it, and in a preferred embodiment, anadditional synchronization field is inserted at the beginning of eachpackage. The synchronization field has a pre-defined pattern of bitsthat can be identified easily by the decoder to be interpreted as thebeginning of the data package. An example for such a synch field is afield with all the bits set to ‘1’.

If data is encoded with a powerful error correction code, such as the(850,485,79) binary BCH code, the synchronization field can be avoided,allowing for a slightly higher baud rate. In the case of such a code,the decoder can loop over the detected fields, trying to start themessage with each field as if it was the start field. The errorcorrection code ensures that the message can only be decoded correctlyif the correct field is selected to be the start of the message. Themain disadvantage of such a code is the relatively high consumption ofCPU resources during decoding, since a worst-case scenario, that ofstarting scanning from the second field, involves making twenty sixattempts at decoding the message. Therefore, a synchronization field ispreferred, even when using such a BCH error correction code.

In a preferred embodiment, decoding is implemented in hand held mobiledevice 16. Device 16 is intended to be small and simple and thus needsto be able to decode the data with minimum usage of memory and CPUresources. In one embodiment, data is scanned in and decoded directly.In another embodiment, however, decoding is implemented in two phases. Afirst phase involves data being sampled and stored in memory and asecond phase involves the stored data being processed and decoded.

Reference is now made to FIG. 3, which is a simplified flow diagramillustrating a preferred implementation of a decoding algorithm suitablefor use with embodiments of the present invention. In the implementationof FIG. 3, decoding is done in the following steps:

-   -   Rough detection of the data window position over the video frame        S20    -   Accurate detection of the data window position over the video        frame S22    -   Bit reconstruction S24    -   Back correction of window position S26    -   Error correction decoding S28

Preferably, the data is decoded in as high an SNR environment aspossible, which means first of all that the position of the data windowneeds to be located as accurately as possible. Therefore, in a preferredembodiment, the detection algorithm focuses on fine positioning of thedata window within the stream of samples. As soon as the window positionis accurately detected, the data can be compared synchronously toavailable wave patterns, enabling highest detection probability.

Stage S20 comprises a rough positioning of the data window over thevideo flame. The goal of stage S20 is to find an inaccurate position ofthe data in the stream of samples, while using minimum memory and CPUresources, and preferably also to make the positioning determination inthe minimum amount of time.

The rough positioning finds the point in which the energy of the datasignal is maximal. The algorithm looks for such energy maxima over apre-defined number of samples, which between them represent at least oneframe of video display.

In a preferred embodiment, line energy is defined by the sum of therelevant frequencies following DFT (Discrete Fourier Transform). If FSKis used, the energy is the sum of two DFTs, for the two relevantfrequencies. It is noted that in order to save computation, it ispossible to change the order and sum the lines prior to calculation ofthe DFT transform.

In the following, the term “window energy” refers to the sum of thewindow line energy. In a preferred embodiment, a rough locationalgorithm comprises sliding a sampling window over the data samples, andthen calculating the window energy for each window location. In order tosave CPU resources, the window is moved over pre-defined intervals, theinterval being relatively large, both in the X and Y-axis. Preferably,sampling is carried out in units of two megaHertz and the windowscanning interval is 12 samples. Thus, the final position as calculatedby stage S20 is inevitably inaccurate as it is dependent upon themovement interval. Preferably, even for rough location, the slidingwindow positions overlap to some extent, so as not to miss the actualmaximum altogether.

A further embodiment saves CPU resources in the rough location stage bydropping some of the scan lines within the current window whencalculating the energy for the window. In a preferred embodiment, linesare processed in an interval of 4 so that only one quarter of the windowis actually used for energy calculation.

Detection stage S22 comprises an accurate positioning of the data windowover the video frame. The input for stage S22 is the inaccurate positionobtained in stage S20. The data samples that are used for the secondstage may conveniently be taken from a field in a video frame thatfollows the frame that was used for the first stage. The second stagemay use a frame that follows directly or indirectly from the frame usedin the first stage.

In a preferred embodiment, the algorithm knows the video rate inadvance. It is therefore able to calculate the position of a sample,that is to say a timing in terms of video scanning that represents acertain location on the frame, from a sample position that representsthe same location in a previous field.

Since both the video rate and the sampling clock have some imprecision,it is preferable to allow a certain inaccuracy in either the video rateor the sampling clock.

Knowing the inaccurate position of the data window in a previous framefrom stage 20, the algorithm predicts the inaccurate position of thewindow in the current field as being substantially the same position.Then, a second window is defined by adding margins to the data window.The second window is bigger than the data window but is also placed inthe inaccurately predicted location. The algorithm stores data extractedfrom the second window in memory for processing, which can be left untillater with fewer timing constrains. As soon as the scan of the secondwindow is stored in memory, the algorithm moves a data size window overthe scan to find the position having the maximum energy. At this stagethe movement interval in both axes is one sample. In a preferredembodiment, in order to save CPU resources, the algorithm looks for amaximum in one axis (Y, for example) and than for a maximum in theorthogonal axis.

Stage S24 comprises recovering the encoded data bits from the sampledvideo signal. Stage S24 may be performed more than once if the data isembedded in more than one video field. The input for stage S24 is theaccurate position of the data window as retrieved in stage S22.

In a preferred embodiment, refining of the data window positionautomatically starts stage S24. That is to say recovering of data bitscan start as soon as the energy is maximized. Such an embodiment allowsfor some inaccuracy in the video rate or sampling clock. The positionrefining in this embodiment may be carried out in a way similar to theprocess described above as stage S22 except that the window margins canbe different in this case since the input position is more accurate.

Data bit recovery from the window comprises looking for a frequencyhaving maximum energy for each line in the data window. The symbolrepresented by the maximal energy is selected as the raw data. In apreferred embodiment, in which only two frequencies are used (FSK), thealgorithm actually compare energies of two frequencies and selects ‘1’or ‘0’ in accordance with the frequency giving the higher energy.

In one of the embodiments referred to above, each data symbol isrepresented by more than one line. In such a case the relevant lines aresummed prior to making the energy calculations. In a preferredembodiment, two sequential lines represent a symbol, and therefore thetwo lines are summed prior to energy comparison.

The line energy in a certain frequency may be calculated by DFT or FFT(Fast Fourier Transform). However, in a preferred embodiment, comparisonis made between the Real part of the DFT rather than its magnitude.Doing so when the DFT window is synchronized to the data window is moreefficiency and more accurate since what it amounts to is that phaseinformation is taken into account.

The present embodiment enables decoding of the data even when noiseenergy is greater than the signal energy. However, a difficulty thatarises with the present embodiments is that it makes it difficult tocorrectly locate the data window in the first place. Thus the algorithmof FIG. 3 is likely to mistakenly position the data window to start atan incorrect line. A preferred embodiment thus attempts to read bitsfrom above and below the predicted data window. In the next algorithmstage, when the window position is refined, the spare bits are alreadypresent, should it be decided that they are part of the window afterall, and no additional reading is necessary.

Preferred embodiments are robust to use conditions, and robustnessincludes enabling quick detection of messages even if the sensor isoriginally pointed in the wrong direction and only subsequently is movedtowards the TV. In such a robust embodiment, the algorithm continuouslywatches for data window energy. As long as the device points to realdata, energy that is received by sequential fields is more or lesssimilar and step S24 leads to step S26, discussed below. Otherwise,energy variation in sequential fields is large and the algorithm breaksits normal flow and loops back to step S20.

In a preferred embodiment, an MLS (Mean Least Square) stage S26 refinesthe data window positions of the previously detected fields, using thepositions retrieved by the previous stages S20 and S22. One way ofrefining the windows position is by using an MLS process. Since framesare equally spaced in time, it is possible to represent the beginning ofdata window by a linear equation such as:s=a*n+b

Where n stands for the number of the field and s stands for the numberof the sample that starts the data window of the field n. a and b arelinear equation parameters.

In a preferred embodiment, the MLS process is activated recursively,deleting points that fall far away from the linear line betweeniterations, until a low standard deviation line is achieved. Using theMLS output, data window positions are updated.

If the actual position of the window is different from the positiondetected by a bit recovery process, than the bits collected from aboveor below the previous window positions are used to correct the mistake.

In a preferred embodiment, a stage S28 of error correction is included.If data is transmitted repeatedly in cyclic manner, errors can becorrected as soon as all data fields are received, regardless of whichone of the fields is received first. However, as discussed above, inorder to correct the errors and interpret the message, the algorithmpreferably determines the position of the first data field within thesequence as received.

As mentioned above, the encoder may use a synchronization field, forexample a field with all the bits set to ‘1’. Then the decoder sorts thereceived data fields according to the number of ‘1’ symbols in eachfield, and decoding starts with the field that follows the one havingthe maximum number of ‘1’ symbols.

If a synchronization field is not used, the decoder tries to correcterrors starting with one of the fields and if it fails it moves on tothe next field until it succeeds. As mentioned above, such a procedureis expensive in terms of CPU resources.

A preferred embodiment of the decoder 16 is universal, in that itsupports data encoding into a variety of video encoding methods, inparticular NTSC, PAL-A, PAL-B, SECAM, etc. Different video encodingmethods utilize different detection algorithm parameters such as numberof samples in field, video rate, etc.

It is possible to use devices specifically programmed with the differentconfigurations and parameters for the different encoding methods.However, a universal device is preferred. The universal devicepreferably detects the encoding method automatically as follows. Anencoding method is set as default and used in initial scanning. If novalid data is detected using the current transmission method for acertain time, say 5 seconds, the algorithm switches to a different setof parameters trying to detect valid data in a new transmission method,and so-on. Once data is detected, the new method is stored innon-volatile memory, such as FLASH memory, and is used as the defaultmethod for the next activation of the device.

Reference is now made to FIG. 4, which is a block diagram of a preferredembodiment of mobile device 16. Light emitted by the TV display isreceived by a one or more photo diodes 40.

As the user is aware, the television picture comprises pixel positionsscanned in scan lines many times a second. The human eye sees a staticpicture due to the human eye's property of persistence. However thephoto diode 40 merely sees successive pixels being lit up on a darkscreen. The photo diode can obtain data from the screen by simplyregistering the brightness picked up from the screen at any giventiming, the registered brightness indicating the current pixel. Theposition of the pixel is inferred by the timing.

The use of more than one photo diode gives the device greater resilienceto the angle at which is being held towards the screen. The signal isthen amplified and filtered by a band pass filter 42. Next, the signalis sampled and digitized by an A to D converter 44. In a preferredembodiment, a sampling rate for the converter is 2 MHz and the filtercut off frequency is about 700 KHz. Code including algorithms for theembodiment are stored in a non-volatile Flash memory 46, which is readand processed by a DSP 48. In order to improve performance, some of thecode is read from the Flash 46 during start up and stored in faster DSPinternal memory, from where it is subsequently executed. If a validmessage is decoded, then the decoded message is sent from the DSP 48 andprinted out by a Micro Printer 50, also located on the mobile device.

Reference is now made to FIG. 5, which is a system diagram showingencoding and decoding apparatus according to the present invention.Encoding apparatus includes a window definer 50 for defining a datawindow in which the data is to be located within the video signal. Anencoder 52 subsequently encodes the message or data file for embeddingwithin the window. At the decoder end, an optical scanner 54 scans thevideo screen. A window finder arrangement 56 as described above locatesthe data window within the screen, as discussed above, and a decoder 58decodes the data as described.

The skilled person will be aware that a video signal contains a soundcomponent, a visual component and control and timing information. Thevisual component comprises a series of frames that make up the movingimage, and the frame comprises two fields of interleaved scan lines. Forthe purpose of the claims which follow, it is this visual component intowhich the data for optical detection is inserted.

It is appreciated that certain features of the invention, which are, forclarity, described in the context of separate embodiments, may also beprovided in combination in a single embodiment. Conversely, variousfeatures of the invention, which are, for brevity, described in thecontext of a single embodiment, may also be provided separately or inany suitable subcombination.

Although the invention has been described in conjunction with specificembodiments thereof, it is evident that many alternatives, modificationsand variations will be apparent to those skilled in the art.Accordingly, it is intended to embrace all such alternatives,modifications and variations that fall within the spirit and broad scopeof the appended claims. All publications, patents and patentapplications mentioned in this specification are herein incorporated intheir entirety by reference into the specification, to the same extentas if each individual publication, patent or patent application wasspecifically and individually indicated to be incorporated herein byreference. In addition, citation or identification of any reference inthis application shall not be construed as an admission that suchreference is available as prior art to the present invention.

1. Apparatus for encoding data for optical detection within the visual image of a video signal, the apparatus comprising: a defining unit for defining a region within said visual image to carry said data such that said data is detectable by optical detection from said visual image of said video signal, and an encoder, associated with said defining unit, for encoding said data for optical detection into video scan lines within said image region such that said detection comprises optical observation of said scan lines, wherein said data for optical detection comprises barcode data.
 2. Apparatus for encoding data for optical detection within the visual image of a video signal, the apparatus comprising: a defining unit for defining a region within said visual image to carry said data such that said data is detectable by optical detection from said visual image of said video signal, and an encoder, associated with said defining unit, for encoding said data for optical detection into video scan lines within said image region such that said detection comprises optical observation of said scan lines, wherein said data for optical detection comprises a software update for a corresponding decoding device.
 3. Apparatus for decoding data encoded optically within a defined region of a visual image of a video signal, said apparatus comprising: an optical detector for optical detection of said image, an encoded region determination unit for determining, from output of said optical detector, boundaries of said defined region within said scanned image, and a data decoder associated with said encoded region determination unit for decoding data received at said optical detector that is determined to be within said defined region, further comprising a printer associated with an output of said data decoder, for printing out decoded data.
 4. Apparatus for decoding according to claim 3, incorporated within a mobile device.
 5. Apparatus for decoding according to claim 4, wherein said mobile device is a handheld device.
 6. Apparatus according to claim 3, wherein said data for optical detection comprises an identification flag and said printer is configured to make only a predetermined number of printouts per identification flag.
 7. Apparatus according to claim 6, wherein said predetermined number is one.
 8. Apparatus for decoding according to claim 4, wherein said mobile device comprises a mobile telephone.
 9. Apparatus for decoding according to claim 3, comprising an accessory for a mobile telephone.
 10. Apparatus for decoding according to claim 3, further comprising a software updater for using said decoded data for self-updating.
 11. Apparatus for decoding according to claim 3, wherein said data for optical detection repeats cyclically and said decoder is operable to decode data whenever a substantially full cycle has been detected irrespective of where in said cyclical repetition said decoding starts at.
 12. Apparatus for decoding according to claim 11, wherein said decoder is operable to use error correction data within said data for optical detection to deduce a starting point thereof.
 13. Apparatus for decoding according to claim 11, wherein said data for optical detection comprises a synchronization field to provide orientation within said cyclical repetition.
 14. Apparatus for decoding according to claim 3, wherein said data for optical detection comprises at least one position flag to indicate places in said cyclical repetition.
 15. Apparatus for decoding according to claim 3, wherein said encoded region determination unit comprises an entropy summation unit for obtaining summations of entropy over a video image to identify said region as a region having maximal entropy.
 16. Apparatus for decoding according to claim 15, wherein said entropy summation unit is configured to provide a sliding window to move over said image, to calculate an entropy summation for each window position and to identify said region as a window position having a maximal entropy.
 17. Apparatus for decoding according to claim 16, wherein said encoded region determination unit is configured to provide relatively large changes in position of said sliding window between each entropy summation to provide coarse determination of said region.
 18. Apparatus for decoding according to claim 17, wherein said encoded region determination unit is configured to provide perturbations to said coarse determination of said region to achieve fine determination of said region.
 19. Apparatus for decoding according to claim 15, wherein said encoded region determination unit is configured to enable decoding of scan lines within a perturbation range around said region, so that data extracted therefrom can be used if said scan lines are subsequently determined to be within said region.
 20. Apparatus according to claim 17, wherein said encoded region determination unit is configured to use a mean least squares (MLS) to achieve fine determination of said region.
 21. Apparatus for decoding according to claim 15, wherein said entropy summation unit is configured to summate entropy over substantially all scan lines and all frequencies within said image.
 22. Apparatus for decoding according to claim 18, wherein said decoder is configured to correct data decoding according to subsequently carried out perturbations.
 23. Apparatus for decoding according to claim 3, wherein said encoded region determination unit is operable to continue to use a region as detected in previous frames of said image.
 24. Apparatus for decoding according to claim 3, wherein said data for optical detection comprises symbols encoded as frequencies within said scan lines, said data decoder being operable to deduce said symbols from said frequencies.
 25. Apparatus for decoding according to claim 3, wherein said data for optical detection comprises symbols encoded in a plurality of scan lines within said region.
 26. Apparatus according to claim 25, wherein said data decoder is operable to sum a respective plurality of scan lines, and to decode a corresponding signal from said summation.
 27. Apparatus for decoding according to claim 3, configured for scanning image produced by a plurality of video encoding methods, said apparatus being operable to scan for a first method, and if energy detected falls below a predetermined minimal threshold then to restart the scanning process.
 28. Apparatus for decoding according to claim 3, configured for scanning image produced by a plurality of video encoding methods, said apparatus being operable to scan for a first method, and if energy detected falls below a predetermined minimal threshold then to scan for another one of said plurality of methods.
 29. Apparatus for decoding according to claim 28, being configured to retain a last used video encoding method as a default method for initial scanning.
 30. Apparatus for decoding data according to claim 3, said apparatus comprising: a decoder for applying a decoding procedure to said data, data handling logic for controlling said decoder and for outputting decoded data, and a starting position recognizer associated with said decoder for using an output of said decoder to determine a start position of said data, said start position being used as a parameter for said data handling logic so as to ensure that said cyclically repeating data is decoded and output in a correct sequence from said start position.
 31. Decoding apparatus according to claim 30, wherein said start position recognizer is configured to recognize a synchronization field within said cyclically repeating data.
 32. Decoding apparatus according to claim 30, wherein said start position recognizer is an error correction circuit for operating with a cyclic redundancy code. 